Discrete Time Markovian Agents Interacting Through a Potential 

A. Budhiraja * P. Del Moral f S. Rubenthaler * 

June 17, 2011 



Abstract 

A discrete time stochastic model for a multiagent system given in terms of a large collection of 
interacting Markov chains is studied. The evolution of the interacting particles is described through 
a time inhomogeneous transition probability kernel that depends on the 'gradient' of the potential 
field. The particles, in turn, dynamically modify the potential field through their cumulative input. 
Interacting Markov processes of the above form have been suggested as models for active biological 
transport in response to external stimulus such as a chemical gradient. One of the basic mathematical 
challenges is to develop a general theory of stability for such interacting Markovian systems and for 
the corresponding nonlinear Markov processes that arise in the large agent limit. Such a theory would 
be key to a mathematical understanding of the interactive structure formation that results from the 
complex feedback between the agents and the potential field. It will also be a crucial ingredient in 
developing simulation schemes that are faithful to the underlying model over long periods of time. 

The goal of this work is to study qualitative properties of the above stochastic system as the number 
of particles (TV) and the time parameter (n) approach infinity. In this regard asymptotic properties of a 
deterministic nonlinear dynamical system, that arises in the propagation of chaos limit of the stochastic 
model, play a key role. We show that under suitable conditions this dynamical system has a unique 
fixed point. This result allows us to study stability properties of the underlying stochastic model. We 
show that as N — > oo, the stochastic system is well approximated by the dynamical system, uniformly 
over time. As a consequence, for an arbitrarily initialized system, as N — > oo and n — > oo, the potential 
field and the empirical measure of the interacting particles are shown to converge to the unique fixed 
point of the dynamical system. In general, simulation of such interacting Markovian systems is a 
computationally daunting task. We propose a particle based approximation for the dynamic potential 
field which allows for a numerically tractable simulation scheme. It is shown that this simulation 
scheme well approximates the true physical system, uniformly over an infinite time horizon. 

Key words: Interacting Markov chains, agent based modeling, multi-agent systems, propagation of chaos, 
non-linear Markov processes, stochastic algorithms, stability, particle approximations, swarm simulations, 
chemotaxis, reinforced random walk. 
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1 Introduction. 

In recent years there has been a significant interest in agent based modeling for complex systems. Examples 
of such models abound in physical and social sciences and include problems of biological aggregation 
|llj . chemotactic response dynamics [6], self organized networks [9], communication systems [7], opinion 
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dynamics[8j, multi-target tracking [3], etc. See [101 [7] for a comprehensive list of references. One popular 
model for interacting multi-agent systems (see [10] and references therein) consists of a large number 
of particles that influence each other indirectly through a dynamic potential field and can formally be 
described through the following system of equations. 

dXi{t) = Vh(Xi(t),t)dt + dWi(t), Xi(0) = Xl G R d , i = 1, • • • N. 

a - 

— h(t,x) = -ah(t,x) + DAh(t,x) + p^2g(Xi(t),x), h(0, x) = h {x). (1.1) 

i=i 

Here Wi, i = 1 • • • N are independent Brownian motions which drive the state processes X{ of the N inter- 
acting particles. The interaction between the particles arises indirectly through the underlying potential 
field h which changes continuously according to a diffusion equation and through the aggregated input 
of the N particles. One example of such an interaction is in chemotactic cell response dynamics where 
cells preferentially move towards a higher chemical concentration and themselves release chemicals into 
the medium, in response to the local information on the environment, thus modifying the potential field 
dynamically over time. In this context, h(t, x) represents the concentration of a chemical at time t and 
location x. Diffusion of the chemical in the medium is captured by the Laplacian in (jl.ip and the constant 
a > models the rate of decay or dissipation of the chemical. Contribution of the agents to the chemical 
concentration field is given through the last term in the equation. The function g captures the agent 
response rules and can be used to model a wide range of phenomenon |10| . The first equation in (jl.ip 
describes the motion of a particle in terms of a diffusion process with drift in the direction of the gradient 
of the chemical concentration. Many variations of this basic model with applications to a wide range 
of fields can be found in [10J. A precise mathematical treatment of (II. lj) presents significant technical 
obstacles and existing results in literature are limited to simulation and formal asymptotic approximations 
of the system. In the current work we will study a discrete time model which captures essential features of 
the dynamics in (jl.ip and is also amenable to a rigourous mathematical treatment. The time evolution of 
the agents will be described through a time inhomogeneous transition probability kernel, where the kernel 
at time instant n is determined in terms of the "gradient of the potential field" at time instant n — 1. The 
agents in turn affect and contribute to the potential field dynamically over time. Thus as in the formal 
continuous time setting described above, the A-agent dynamics is strongly coupled and describes a (time 
inhomogeneous) Markov chain in E N where E (a compact subset of M. d ) is the state space of a typical 
agent. Although the model description is considerably simpler in discrete time, our objective here is to go 
beyond formal heuristics (as is the current state of the art for the continuous time model in (jl.ip ) and to 
formulate and study precise mathematical properties of the system. We will establish convergence of the 
stochastic model to the solution of a non-linear dynamical system, over an arbitrary fixed time horizon, as 
the number of agents approach infinity and as a consequence obtain a propagation of chaos result (Theorem 
II. H and Corollary II. 2p . We are particularly interested in the stability of the system as JV-> oo, over long 
periods of time. A mathematical understanding of the stability behaviour is key to the study of long term 
structure formation resulting from the complex interactions between the agents and the potential field. 
Stability results for the system are also crucial ingredients for studying the behaviour of approximate 
simulation schemes over long intervals of time. Denoting by n the time parameter, we will give conditions 
under which, as N — > oo and n — > oo (in any order) the potential field and the empirical measure of the 
A particles converges to limits that are independent of the initial configuration (Corollary II. 5p . These 
limits are characterized as the unique fixed point of the limit deterministic non-linear dynamical system 
(Theorem II .31) - Uniform in time convergence of the stochastic model to the non-linear deterministic dy- 
namics is established as well (Theorem 1 1 . 4[) . In general, simulation of interacting Markovian systems is 
a computationally daunting task. We propose a particle based approximation for the dynamic potential 
field which allows for a numerically tractable simulation scheme. Using the above stability results we show 
that this simulation scheme well approximates the true physical system, uniformly over an infinite time 
horizon (Theorem II. 9p . 
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Before we give a formal description of the model, we list some common notation that will be used 
in this work. For a Polish space S, B(S) will denote the Borel sigma field on S and V(S) the space of 
probability measures on (S,B(S)). For x G 5, 5 X will denote the element in V(S) that puts unit mass at 
the point x. For ^ G P{S) an d a /i-integrable real measurable map / on (S,B(S)), we denote f s fd/j, as 
l-i(f) or (/x, /}. Similar notations will be used for signed measures. For /i G V(S), Sjsr(fi) denotes a random 
measure defined as ^2^=1 > where {£fc}fcLi are i-i-d 5 valued random variables. We denote the space of 
real bounded measurable maps on (S,B(S)) as BM(5) and for / G BM(<S), define ||/||oo = su Pze5 
The space {/ G BM(5) : ||/||oo < 1} will be denoted as B\ (S). For a signed measure \x on (S,B(S)), we 
define the total variation norm of fi as s&Vf^BxCS) Im(/)I an d denote it by ||^||yy. A real function / on 
S is said to be 6-Lipschitz if, for all x,y G 5, |/(ic) — /(y)| < 6 d(x,y), where d is the metric on S. A 
transition probability kernel (also referred to as a Markov kernel) on S is a map T from S x 0(5) to [0, 1] 
such that for all x G S, T(x,-) G V{S) and for all y4 G B{S), T(;A) G BM(5). For / G BM(5) and a 
transition probability kernel 7" on 5, define 7"/ G BM(«S) as Tf(-) = j s f{y)T{-,dy). For a closed subset 
5o of S, n G V(Sq) and a transition kernel 7" on 5, we define /i7~ G "P(5) as fiT(A) = J So T(x, A)fj,(dx). 
For d > 1, M. d will denote the (i-dimensional Euclidean space, the standard norm on which will be denoted 
by | • |. N [resp. No] will denote the space of positive [resp. nonnegative] integers. Cardinality of a finite 
set G will be denoted by \G\. 

1.1 Model for Interacting Markovian Agents. 

The system consists of particles whose states at time k G No denoted as X\(k), ■ ■ ■ Xw(k) take value 
in a compact set E C W 1 with a non-empty interior. Given that the j-th particle is at location x at 
time instant k — 1, it transitions, independently of other particles, to a set A C B{E) with probability 
M r,k - 1 {x, A). Here M^- 1 is a transition probability kernel determined by a nonnegative function r]k-i 
which represents the "potential field" at time instant k — 1. Specifically, for any nonnegative function 
^ : E — > M + , the Markov kernel is defined as follows. 

M 9 (x, dy) = Q(x, dy)e- x ^ {x ^^+ + Q (x, dy) (l - J Q(x, dz)e~ x ^-^+^ . (1.2) 

Here A G (0, oo) and Q, Qq are two transition probability kernels on E. Roughly speaking, to generate a 
sample from M 9 (x, •) one follows the following steps: 

• A sample Y is drawn from Q(x, ■) and a sample Y is drawn from Qo(x, •). 

• If ^(Y) > ^/(x) the sample point Y is accepted. This corresponds to preferential motion in the 
direction of the gradient of the potential field. 

• If ^f(Y) < ^f(x) we accept the sample point Y with probability e - ^*^ - *^)) and take Y with 
probability 1 - e -A(*(*)-*(y))_ 

The kernel Q captures particle dynamics in absence of the potential field, while Qo can be regarded as the 
perturbation to the nominal dynamics against the concentration gradient, caused by the potential field. 
Note that the effect of Qq increases as A becomes larger. 

We now describe the evolution of the potential field and its interaction with the particle system. This 
evolution will aim to capture the essential features of the PDE in (jl.ip which are: diffusion, dissipation and 
dynamic agent input. Let P and P' be transition probability kernels on M. d having density with respect 
to some fixed reference measure t on M d (for example, the Lebesgue measure). Throughout, we will speak 
of densities as with respect to I. Given m G V(E), define a transition probability kernel R m on M. d as 

Rm(x, A) = (1 — e)P(x, A) + emP'(A), x G M d , A G B(M. d ). (1.3) 
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Then given that the potential field at time A; — 1 is described by a nonnegative function rjk-i on M. d 
satisfying f Rd rjk~i(x)dx = 1, and the state values of the N particles are Xi(k — 1), i = 1, • • • N, rjk is 
defined by the relation. 

1 N 

Vk(y) = Vk-i(x)R mk _ 1 (x,y)£(dx), m k ^ = — V^(it-i). (1-4) 

In this description, diffusion and advection of the chemical is captured by the kernel P, dissipation by 
the factor (1 — e) and contributions by the agents through the term em^-iP' '. We remark that unlike the 
continuous time setting, here we introduce a single parameter e rather than two distinct parameters a and 
f3. This parametrization ensures that the concentration function rjk, for each A; is a probability density 
on R d . The more general setting can be considered as well (although not pursued here) by considering 
suitably normalized concentration fields. 

Thus summarizing, analogous to (jl.ip , the coupled system of equations describing the evolution of the 
potential field and particle states is given as follows. Denote by V*(R d ) the space of probability measures 
on M. d that are absolutely continuous with respect to I. We will identify an element in V*(M. d ) with its 
density and denote both by the same symbol. For N > 1, define IT^ : E N — > V{E) by the relation 

1 N 

u N ( yi ,--- yN ) = -Y, s y^ (yi,---y N )eE N . (1.5) 

i=l 

Fix random variables X\, • • • , Xjy with values in E and rj^ £ V*(R d ). Let uIq = H N (X\, • • • , Xn). The 
interacting system of particles and the potential field is described as a family (X N (k),m^ ,rj^)k £ ^ of 
E N x V(E) x V*(R d ) valued random variables on a probability space (f^J 7 , P), defined recursively as 
follows. Let X N (0) = (Xi, ■ ■ ■ ,X N ) and define J" = ^(0)}. For k > 1 

'r(X N (k) G A | 73k-i) = ®f=i (^jv (fc -i)^ e (A), A E B{E N ) 

1 r 



J=l \ U X? (fc-1) 

■ v - n^(x^)) = ^E,?=i^ 



my = 11" (A •(/■■) ) ■ - -V > . ; " , <)y.X 

.N ,N 

~"'k-l 



., AT 



J- fe = a(r 7 f,X JV (A;))Vj- fe _ 1 



Along with the N particle system we will also consider the non-linear Markov model which formally 
corresponds to the N ->■ oo limit of (JOJ). Define the map $ : 'P(-E) x V*(R d ) -> V{E) x V*(R d ) by the 
relation 

$(m,r/) = (mM^,7]R m ), (m,rj) E 7>(E) x P*(R d ). 

Define a sequence of probability measures (m n ,rj n ) on R d by the following recurrence formula. Fix 
(m ,r/ ) E 7>(.E) x ^*(M d ). For n E N 

m n+1 = m n M^ , (1.7) 

Vn+l = VnRm n ■ (1.8) 

In other words, 

(m n+ i,ry n+ i) = $(m n ,rj n ) . (1.9) 
Such a coupled system is similar to what can be found in [2j [3] . 



1.2 Main Results. 

We now summarize the main results of this work. For measures (m,rj), (m! ,rj') E V{E) x 'P*(lR c( ), we 
define the norm 

\\(m, rj) - (m', rj')\\ = \\m - ru'Wrv + \\v - v'Wtv ■ 
We begin with an assumption on the kernels P, P'. 
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Assumption 1. The kernel P' is uniformly bounded on M. d x M. d and P'(-,y) is Lipschitz, uniformly in 
y E R rf , namely, for some Mp>,lp> E (0, oo) 

sup P'{x,y) < M P >, sup \P'(x, y) — P'(x' , y)\ < lpi\x — x'\, Vrr, x' E M. d . 

(x,y)eR d xR d y£R d 

The kernel P satisfies, for some Mp E (0, oo) 7 

sup / P(x, y)l{dx) < M P . 

y£R d Jx£R d 

The following result establishes the convergence of the stochastic system to the non-linear deterministic 
dynamical system over any fixed time horizon. 

Theorem 1.1. Suppose that Assumption^ holds. Also suppose that 

W^o ~ ^olloo — > and sup E(| (mo — thq , f)\) — > as N — > oo. 
feBi(E) 

Then, Vk E N , 

sup E(\(m% -m k J)\ + \\r£ -rikWoo) — > 0. (1.10) 
/GBi(£) N-^+oo 

We remark that E||r/^— ?7fe||oo — ^ implies, by an application of Scheffe's theorem, that E||r/^— r/fc 1 1 T 1 V ~~ * 
0. This observation will be used in the proof of Theorem 11.41 

As an immediate consequence of this result we obtain the following propagation of chaos result. For 
p > 1, denote by n\ ,p the probability law of (X± (k), ■ ■ ■ (k)) on E p . 

Corollary 1.2. Under Assumption^ for every m k' P converges weakly to m® v , as N — > oo. 

We now study time asymptotic properties of the system. We begin with two basic assumptions. The 
first is on the kernel Q. 

Assumption 2. There exist e Q E (0, 1) andi x E V(E) such thatMx E E, V A E B(E), Q{x,A) > e Q £i(A). 

It is well known that under Assumption [2] Q is (1 — eg)-contracting for the total variation norm (see 
Lemma |3. II in Appendix for a proof), i.e 

HmQ - m'QIItv < (i - e Q )\\fi - h'Wtv^ij! g V{e). 

Next, we will make the following assumption on the kernels P, P'. 
Assumption 3. 1. There exists j3(P') E (0, 1) such that for all m,m' E V{E), 

\\mP' - m'P'\\ TV < (3(P')\\m - m'\\ T v- 

2. The kernel P is uniformly bounded on M. d X M. d , i.e. for some Mp E (0, oo) 

sup P(x,y)<M P . 

(x,y)£R d xR d 

We begin with the following result on the fixed points of the dynamical system (|1.9p . 

Theorem 1.3. Suppose that As sumptions [Jl\^ and\3\ hold. Then there are Ao,£o G (0, oo) such that for 
all A < Aq and e < eo, $ has a unique fixed point in V{E) x V*(R d ). 



5 



Recall the sequence (m n ,r] n ) denned through equation (jl,9p recursively with a fixed initial pair 
(too,?7o) £ 'P(E) x V*(R d ). Also recall the collection of random variables (X (k),rriu ,T]u)k& with 
values in E N x V(E) x V*(R d ) defined in ([12]) starting with a fixed r]ff £ V*(R d ) and X l5 . . . , X N G E. 

We now consider uniform in time convergence of the stochastic system to the non-linear dynamical 
system. For that we will make the following additional assumptions. 

Assumption 4. 1. P(-,y) is Lipschitz uniformly in y E R d namely, for some lp £ (0, oo) 

sup \P(x,y) - P(x',y)\ < l P \x - x'\, Vx,x' G R d . 

j/GK d 

2. There exist p £ V(R d ) and l P , such that \/x,x' £ E, A E B(R d ), 

\P'{x,A) - P'(x',A)\ < l P ,\x-x'\p{A). 

3. P(x, •), P'(x, •) are Lipschitz uniformly in x £ R d namely, for some Ip.p/ £ (0, oo) 

sup max{|P(x,y) - P(x, y% \P'(x, y) - P'(x,y')\} < l P>p/ \y - y'\, Vy,y' £ R d . 

x<m. d 

4- For some M PP i £ (0, oo) 

sup max{ / P(x,y)l(dx), / P' \x,y)l(dx)} < Mp y pt . 

Denote by £g the restriction of the measure I to E. 

Assumption 5. For all x £ E, Q(x, .) and Qo(x, .) have densities with respect to £e- The densities are 
bounded on E x E, namely for some Mq^q q £ (0, oo) 

sup m&x{Q(x,y),Q (x,y)} < M QiQo . 

(x,y)eExE 

Furthermore, for some Iq,q £ (0, oo) and p £ V(E) 

m&x{\Q(x,A) - Q(x',A)\,\Q (x,A) - Q (x',A)\} < l Q>Qo \x - x'\p(A) \fx,x' £ E,A £ B(E). 

Theorem 1.4. Suppose that Assumptions [7][2l El \J\ an d El hold. Let eo,Ao be as in Theorem \1.3\ Then, 
whenever e < cq and A < Ao, we have: 

(i) For some Co £ (0, oo), 

V5 > , 3Nq, no £ N such that Vn > no and N > Nq , sup E(| (m^ — m n , f)\ + \\Vn ~ VuWtv) < erf . 

ll/l|oo<l 

(ii) If \\t)q — rjoWoo — > and supj GKl ( E ) E(| (mo — , /)|) — )• as N — > oo, then for some c\ £ (0, oo) 

V5 > 0, 3N £ N such thatMn and N > N , sup E(|(m^ - m n ,/)| + ||n^ - t/„||tv) < erf. 

Il/Il»<i 

As an immediate consequence of Theorems 11.31 and 1 1 . 41 we have that under suitable conditions (rriu , n^ ) 
approaches the unique fixed point of as k — > oo and A — > oo. Namely, 
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Corollary 1.5. Suppose that Assumptions [i][H El an d El hold. Let eo,Ao be as in Theorem \1.3[ Fix 
e E (0, €q) and A G (0, Aq). Denote the corresponding unique fixed point of & by (moo, 7700). Then, 



lim sup lim sup sup E(|(m n - moo, /)| + ||r/„ -T7oo[|rv) 

n->oc Ar->oo ||/||cx><1 

= lim sup lim sup sup E(|(m^ - moo, /)| + \\Vn ~ V°o\\tv) 

N-^oo n->oo Il/H^^l 



Simulation of the stochastic system in (II. 6p or numerical computation of the paths of the deterministic 
dynamical system (jl,9p can in general be quite hard and so it is of interest to develop good approximation 
schemes. One flexible and appealing approach is to approximate the measures, 77^ in the first case and the 
measures (rj n , m n ) in the second case, by discrete probability distributions through a collection of evolving 
particles. We will consider one such particle scheme in this work and show that, under conditions, the 
error between the dynamical system in (jl.9p and the one obtained through a particle approximation can be 
controlled uniformly in time. We will also prove a similar result for the error between the actual physical 
stochastic system and the one obtained through particle approximations of 77,^. For these results we will 
make the following additional assumption. 

Assumption 6. P, P' are Gaussian kernels and £ is the Lebesgue measure on M. d . 

Note that if P, P' satisfy Assumption [6l then they also satisfy Assumptions Q] [3] and U] (for Assumption 
[3] see Lemma [3.6p . 

We propose the following particle system for the system (|1.9p that starts from (mo, 770) G V(E) x 
V*(R d ). We assume that we can draw samples from (mo,7/o)- We remark that although for Gaussian 
kernels, the integral in (|1.4|) can be computed analytically as a mixture of Gaussian densities, the com- 
putation becomes numerically unfeasible since the number of terms in this mixture grows linearly over 
time. 

Denote by (Xf (0), • • ■ X$(0)) a sample of size N from m . Let m^ = IF^Xf (0), ■ ■ ■ X$(0)). The 
approximating particle system is given as a family (X N (k), , rj^)ke^n °f E N x V(E) x V*(R d ) val- 
ued random variables on some probability space (Q,J-, P), defined recursively as follows. Set X^O) = 
(Xf (0), • • • X%(0)), fff = r?o, Jo = a(X N (0)). For k > 1 

¥(X N (k) G A I F k ^) = <g)f =1 (fc _ 1} M^-x) (A), A G B(E N ) 

~ N _ 1 v^A r r 

m k ~ N L,j=l °Xf(k) (in) 

^ = (1 _ c )(^(d)P) + eCmJLiP') 

Here S N (f]^_ 1 ) is the random probability measure defined as 5^i=i ^y.^fc) where Y" 1 Ar (A;), • • • Yff (k), 
conditionally on J-k-i, are i.i.d. distributed according to fju_i- 

Notice that under the above Gaussian assumption, 77^ is a mixture of 2N- Gaussian random variables 
for any k > 1, so we can compute numerically its density at any point. So (11. lip defines an implementable 
particle scheme. We first consider convergence over a fixed time horizon. 

Theorem 1.6. Under Assumption^ we have for all k, 

sup E(|(mf -m fc ,/)| + ||77f - mhv + WvZ -%IU) — > 0. (1.12) 

/GBi(B) N^+oo 

We now consider uniform in time convergence of the approximation scheme. 
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Theorem 1.7. Suppose that Assumptions^ [5] and{6\hold. Then there exists c\ G (0, oo) such that for 
e < e o> A < Ao and (i]o,mo) G "P*(R d ) x V{E), the sequences defined in 111. 11)) are such that 

\/5 > , 3Nq such that \/n and N > Nq , sup E(| (m„ — m n , /) | + \\fj^ — r] n \\Tv) < c\8 . 

feBi(E) 

As corollaries of Theorems II .1) [L^l 11.61 and 11.71 we have the following results. 
Theorem 1.8. Under Assumption^ and assumptions of Theorem \l.ll we have for all k > 0, 

sup E(\(mZ-mj<J)\ + \\T£-i£\\Tv)-> 0. 

fGB^E) iV^+oo 
Theorem 1.9. Under assumptions^ [5] and{5\ there exists c\ G (0, oo) such that for e < eo, A < Ao, 
V5 > 0, 3N ,n G N such thatMn > n and N > N , sup E(|(m„ - fh„ , f)\ + ||r/^ - ^||tv) < c\8 • 

ll/l|oo<l 

We remark that in approximating % by fju and by we have taken for simplicity N particles 
for both approximations. Although not pursued here, one can similarly analyse particle schemes where 
the number of particles for approximating rj}~ is different from that used for approximating mfc. 

2 Proofs. 

2.1 Convergence over a finite time horizon. 

In this subsection we prove Theorem 11.11 and Corollary 11.21 We begin with a lemma that will be used 
several times in this work. Proof is immediate from Ascoli-Arzela theorem. 

Lemma 2.1. Let K be a compact subset of M. d and let for a, b G (0,oo), F a ^{K) be the collection of all 
functions f : K — > M such that ||/||oo < a an d f *s b-Lipschitz. Then for every 5 > there exists a finite 
subset F^ b (K) of F a ^(K) such that for every signed measure n on K 

sup | (//,/)|)< max \(fj,, g)\ + 6\\^\\tv ■ 
feF a!b (K) 9£F S a , h {K) 

Frequently, when clear from context, we will suppress K in the notation when writing F,F S . The 
following elementary estimate will be used several times. 

Lemma 2.2. For all x, y, x', y' G R, 

< A|x — x'\ + X\y — y'\ . 



-\(x-y)+ _ e -X(x'-y')+ 



Proof of Theorem li.lt We proceed recursively. Note that, by assumption, f 1 1 . 1 [) holds for k = 0. Suppose 
now that (|1.10p holds for some fixed k G No- Then 

m% +1 - m k+1 = mf +1 - mf M< + AP? - m^M Vk + M % - m fc M % . 
From Lemma [3.51 in the Appendix 

sup E(|(mf +1 -mfM^,/>|)<-5L —> 0. 
/eBi(S) VA AT->+oo 
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Note that for all / G Bi(E), m G V(E) and 77,77' G P*(M d ) 
|mM"(/)-mM' 7 '(/)| 

/ m(dx)Q(x,dy)(e~ A ( r 'W-^)+ - e - A (V(*Wto))+)_/-( y ) 
+ j m(dx)Qo(x,dy) 

x ( f Q(x,dz){-e- XMx) - r '( z)) + + e A fo'0 B WW)+ ) f(y) 
\JzeE J 

Thus by Lemma 12.21 and since (|1.10p holds for k, we have, V/ G B\(E), 

E(\(m»M*£ -m^M*,/)|) <4AE(||77f -TyfelU) — > 0. 

Also, for / G 

(m^M^-m k M^,f) = (m%-m k ,M^f). 

Since / G £i(P), we have that M Vh f G #i(P) and so 

E(\(m k M^ -m%M^J)\) < sup E(|(m fc - m%,g)) . 

geBi(E) 

Once again using the fact that (jl.lOp holds for k, we have from the above inequality, that 

sup E(|(m fc M % - M Vk , f)\) — ► 0, 

/GSi(£) JV-H-00 
and combining the above convergence statements 

sup E(|(m^ +1 - m k +ij > 



/efii(B) 



0. 



Next 



r?f +1 (x) - 77 &+1 (x) = (1 - e)(P* (r/f - Vk ))(x) + e(P' * (m£ - m fc ))(x), s G R d , 

where for /i G V(R. d ), P-k fi is a function on R d defined as P -k ji{x) = f™ d P(y,x)fj,(dy), x G 1 
/J G V{E) is defined similarly. Using Assumption [T] we have 

E (||P *(^-%)|| 0O )<MpE (11^-77,11^) —> 0. 
Finally, for an arbitrary 5 > 0, we have from Lemma l2.1l and Assumption [T] that 



P*(mf -m fc )|| c 



sup 



(m k (dx) - m k (dx))P'(x,y) 



(2.1) 



P' * // for 



< 



max j (m, — m k , g) | + 25 



gen 



where F s is the finite family as in Lemma 12.11 associated with K = E. Recalling that (jl.lOp holds for 
k and noting that 5 > is arbitrary and the family F s (Mp/,lpi) is finite, we have from the above two 
estimates that 



n\\vk+i-m\ 



00 — > 0. 



The result follows. 



□ 



Proof of Corollary 1 1.21 We can apply Proposition 2.2 (i) p. 177 of |12] to get that Vfc,p G N, V</>i, . . . ,<p p G 
C b (E), £(77^(01 ® • • • ® P )) — >■ m® p ((/)i ® • • • We conclude by a denseness argument. □ 

N— >+oo 
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2.2 Existence and Uniqueness of Fixed Points. 

In this section we will prove Theorem 11,31 Some of the techniques in this section are comparable to what 
can be found in [3]. 

For g : R d — > R, let osc(g) = sup^, ye ^d \g(x) — g(y)\. For n > 1, define <3? n recursively as <E> n = ^o^™" 1 , 
where 3? 1 = <£. Let M pP / = max{ M P ,M P /}. We begin with the following lemma. 

Lemma 2.3. Suppose that Assumptions^ and\3\hold. There are eo, Ao G (0, 1) such that for any e < eo 
and A < Ao there exists 9 £ (0, 1) such that Vn 

\\($ n (m , m )-$ n (m' ,r,' )\\<4e n -\ 

for all (m ,7?o),K)^o) G ^(^) x V*{R d ). 

Proof. Note that from Assumption [3] (1), for any (m, 

||r/P m - rj'R m ,\\TV < (1 - e)||r/ - t/HIV + e/3(P')ll™ - ro'||ZV . (2.2) 

and by Lemma [3.21 in the Appendix, 

WmM"! _ m'M^Wrv < (1 - ege" 



-AOSC(^) ) || m _ m /M _ 



(2.3) 



Fix (m ,77o),(m ,?7 ) £ P(-E) x and let (rn n ,T} n ) = <S> n {m ,r} ), (m' n ,rf n ) = $ n (m ,?? ). From 

Assumptions [1] and [3l we have Vn > 1 , 

osc (r/ n ) < Mp t pi , osc(r7 n ) < M pP / . (2-4) 

and therefore for k > 2, 

||m fc _iM^-i -m^M^-iHrv < (1 - e Qe - AM ^')IK-i - m'^TV- (2.5) 
For k > 2 the measures mfc_ 2 P' and m' k _ 2 P' have densities and these densities satisfy Vy 



m k ^ 2 (dx)P'(x, y) - m' k _ 2 (dx) P' (x , y) 

< M P> pi\\m k -2 ~ "4-2 1 1 TV • 



(2.6) 



The measures %_ 2 P and r]' k _ 2 P have densities as well and these densities satisfy Vy, 



|r/ fe _ 2 P(y)-^_ 2 ^(y)| 



{m-2{dx)P{x, y) - r]' k _ 2 (dx)P(x, y)) 



< M PiP / ||?7 fe _2 - r7fe_ 2 ||ry 



(2.7) 



So, Vx, y G M d , using flO}, ^1$, fl2 

e -A(»7fc_x(a!)-»7 fc _i(j/)) + _ e -A(»7j._ 1 (x)-?4_ 1 (i/)) + 



< A|?7 fc _i(a;) - r/^x)! + A|%_i(y) - r^-iClO! 

< 2AMp )P /((l - e) || 2 - ??fc- 2 ||TV + 4 m k-2 ~ m' k _ 2 \\ TV ) . (2.8) 

By dHJ we then have, for / G &x{E), 

\m k „ x M^(f) - m k „ x M<-^f)\ < 4AM PjP /((l - e)|| % _ 2 - ^_ 2 ||w + e||m fe _ 2 - m' k _ 2 \\ TV ) . (2.9) 
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By (|23|) . UM, we get 

\\ m k -m' k \\ T v < \\m k ^iM rik - 1 - mk-iM^-iWrv + \\mk-iM n ><-i - m'^M^-i \\ TV 
< 4AMp )P /((l - e)||r? fe _2 - f] k _ 2 \ T v + e\\m k -2 - m' k _ 2 \\ TV ) 

+ (1 - e Qe - AM ^')IK-i " "4-iIItv , 

and combining this with (|2,2p . we have 

||(mfc,%) - {m' k ,r[ k )\\ < 4AM PjP /((l - e)||?? fe _2 - ti'^Wtv + 4 m k-2 ~ m' k _ 2 \\ T v) 

+ (1 - e Q e- AA/ ^')IK-i - m^iHizv 
+ (1 - - ^LiIItv + e/3(P')ll"ifc-i - m'k-ihv ■ 

(2.10) 

We can find eo G (0, 1), Ao G (0, oo) such that for all e G (0, eo), A G (0, Ao), there exists a = #(e, A) G 
(0, 1), such that 

sup((l-6),(l-6 Q e- AM ^') + 6/3(P')) 4AM P ,p^ i 

Note that 

|| (m , m)-(m' Q , Vo)\\ < . 
||(mi,r/i) - (mi,7?i)ll < 4 - 

We then have by recurrence that for e G (0,eo), A G (0,Ao), ||(m- n ,r/ ra ) — (m^,^)|| < 4# n_1 , Vn G No- □ 

With eo, Ao and #(e, A) = 6 as in the above lemma, let k = 4AA ^ p - p/ , Then from the estimate in ([2.1ip 
it follows that, for all k > 2 and e G (0, eo), A G (0, Ao), 

a k + ran < 6* (a fc _i + na k _ 2 ) , (2.12) 

where a k = \\(m k ,r] k ) — (m' k ,r]' k )\\. As an immediate consequence we have the following corollary which 
will be used in Subsection 12.31 



Corollary 2.4. Suppose that Assumptions^^ and\^hold. Let eo,Ao G (0,1) be as in Lemma \2. 31 Then 
with 9 G (0, 1) as in Lemma \2. 3\ associated with a fixed choice of e < eo, A < Ao, we have, for each k > 2 

\\(m k ,r) k ) - (m' k ,r]' k )\\ +K||(m fc _i,»7fc_ 1 ) - {m' k ^ 1 ,r ] ' k _ 1 )\\ 
< 0*- 1 [{{(mum) - + K\\(m ,Vo) ~ (m' , V ' )\\) . (2.13) 

Suppose further that Assumption^ holds and that mo has a density with respect to £e that is bounded by 
M mo . Then, for k > 1, 

\\(m k , m ) - (m' k , V ' k )\\ < 9 k - l (2 + K + 2\(M mo +M Q , Qo )) (\\(m , m ) - (m' , V ' )\\) . 

Proof. Equation (I2.13D comes directly from (I2.12p , Next note that 

||?7l - ViWtv < \\ m o ~ m o\\TV + ||% - VoWtv ■ 

Also, recalling (|2.ip . 
HmoM 770 -moM^Wrv 



<2 m (dx)Q(x,dy)X(\r] (x) -rj (x)\ + \rj (y) -r]' (y)\) 

J x,y£E 

<2X(M mo +M QiQo )\\ m -r 1 ' \\ TV , 
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and 

\\moM v '° - m' M v °\\ T v < \\™o - m' \\ T v ■ 

Combining these estimates 

||(mi,??i) - (mi,r/i)|| < (1 + sup(l, 2A(M mo + M Q ,Q ))\\(m ,vo) - {m'o,v'o)\\ ■ 

□ 



Proof of Theorem \1.3[ Take eo,Ao as in Lemma 12.31 and fix e G (0, eo) and A G (0, Ao). The uniqueness 
is immediate from Lemma 12.31 For existence, take any (m-o^o) £ 'P(^) x V*(M. d ) and define recursively 
\/k > 1, (mk,r]k) = <I>(mfc_i, rjk-i). We have for all k > 1, p > 1, using the 9 given by Lemma 12.31 



\\$ k +P(mo,Vo)-* k (rn ,r)o)\\ < 4^. (2.14) 

So the sequence (mfc,%) is a Cauchy sequence in (V(E) x || • ||) and thus it has a limit (racci/oo). 

By taking p — > +oo in (|2.14p . we see that {m k ,rj k ) — > (m-oo,^oo)- D 

2.3 Uniform Convergence over the infinite time horizon. 

In this subsection we will prove Theorem 11.41 and Corollary 11.51 



Proof of Theorem \1.4\ Fix 5 > 0. For k G N, we take & k+1 ^° °e a (random) operator such that 
^k+i^k^k) = ( m AH-i> ^fc+i)- ^ V convention, we take $^(7710,%) = { m i ^i)- Let us denote 

for 1 < j , ^(m , ??o) = $f *JLi • • • $f (m , %) and for j = , $^(m , r/ ) = (m , r? ) • 

Also, recall that, for i < j, = $ o • • • o $ (j - i times). We set $(°) = Id . Note that for n > 1, 

n 

W^) " Kife) = 2[ $(n " fc)o$ ^ o ^-i( m 0'^)- $(n " fe)o * °^»-iK.K>)] ■ ( 2 - 15 ) 

fc=i 

We set V/c G N, 

{ m k,k^Vk,k) = ®l:k( m O,Vo), (m k ,k,Vk,k) = $o^- 1 Klo)- 

Then 

n-1 

(m£,^) - (m m% ) = £ ($("- fe - 1 '« w ,i +1 ) - ^"V^+i^M+i)) 

fc=2 

"I" {{ rn n,ni r ln,n) ~ ( m n,n, Vn^n)) 

+ (V^ViUf ) - * (n_1) (mi,^)) • (2.16) 

Notice that 

Vk,k = Vk,k for al k > 1. (2.17) 
Now fix a fc G {2, . . . , n — 1}. The signed measure r)^ k+1 — rfk t k+i has the following density 

(Vk,k+l - Vk,k+i)(y) = e / (mf (dx) - m k _ k (dx))P'(x, y), y £R d . 

JxGE 
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From Assumptiong] (2), for all / G Bi(R d ), 



P'(x,dy)f(y) 



is ip/-Lipschitz. By Lemma 12.11 we then have, for every / G Bi(M. d ), 5 > 0, 



Vk,k+i(f) ~ *7k,fc+l(/)| 



i6E 



(m kik (dx) - m k ,k{dx)) / P'(x,dy)f{y) 



< e ( 2<5+ sup |(m^ fe -m k!k ,g) 



(2.18) 



where Fj 5 = Ff- (F). Now by Lemma [331 with Ci(5) = \Ff 



mVk ik+ i-Vk,k+i\\Tv)<2e(S + 



Ci(S) 



(2.19) 



Using (I2.17P once again 
For any / G B\(E), we have 



m N 
m k,k+l 



m Kk+1 = m£ fc M"M - mfc fc Af».i 



N 

k,k+l 



m k ,k+i, f) = (m^ k - m k ,k, M^ k f) . 



(2.20) 



From Lemmas 13.31 and 13.41 in the Appendix we see that the function x G E i— > M rik ' k f(x) is ^Qo-Lipschitz 
and bounded by 1 where /q,q = Iq,q (3 + 2Xlpp/). So using Lemma [2.11 once again, we have, for 5 > 0, 



E (ll r "fc,fc+i - m fc,fc+illr\/) < 25 + E(sup \(m ktk -m k<k ,g) 



(2.21) 



where F| = Ff - (F) and C 2 (<5) = |F 2 5 |. We will now apply Corollary 12.41 Note that, from Assumption 
for every k, m kk+ i has a density on F with respect to Ie that is bounded by 2Mq j q . This, in view of 



Corollary along with (l2~T9D and ([2^T]) yields Vfc G {2, . . . , n - 2}, 



E 



TV 



< C0' 



n-fc-2 



45 + 



Ci(<5) + C 2 (<5) 



where (7 = (2 + k + 6AMq 5 q ). Note that the above inequality holds trivially if k = n — 1. For the term 
in the second line of fj2. 16f> . note that, for n > 1, 

2 

sup E(|(m£ n - m n , n ,/)|) < -7= , r/^ n = r? n) „ . 
f£Bi(E) VN 

The norm of the term in the third line of (|2.16p . using Lemma [2.3[ can be bounded by 46 n ~~ 2 . Combining 
these estimates, for all n > 1 



sup E(|(m n - m n , f)\ + \\rj n - r] n \\ TV ) 
/€Bi(B) 

n-1 



»-/--2c [ 45 + giW+gaW ') + 4^-2 



< -= + C 45 + 
x/iV 



Ci((5) + c 2 ((5)\ e- 1 



+ 49 



n-l 



The result now follows on combining the above estimate with Theorem ll.il 



□ 
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Proof of Corollary ] 1.51 Fix 5 > 0. From Theorem 11.31 there exist (m^r/co) £ V(E) x "P*(R d ) and no 
such that Vn > no, 

) - ("100,7700)11 < 

From Theorem 11.41 there exist Nq,tii £ N such that Vn > \/N > Nq, 



And so 



sup E(|(m n - m n ,f) \ + [|t/„ - r? n ||tv) < 5. 

/6Bi(B) 



limsuplimsup sup E(|(m„ — moo,/)| + ||n„ — tJooWtv) < 2£, 

n->+oo JV^+oo f£Bi(E) 

limsuplimsup sup E(|(m n — m-oo, /)| + ||n„ — ?7oo||tv) < 2<5 . 

7V^+oo n->+oo /eBi(E) 



□ 



2.4 Proof of Theorem \TM 



In this subsection we will take Assumption [6] to hold. Recall that under Assumption [6j we have that 
Assumptions Q] [3] and 2] hold automatically. 

Proof of Theorem \1.6l We proceed recursively. Since fj^ = r/o, we have using Lemma [33] that (|1.12p holds 
for k = 0. Suppose now that (|1 . 12j) holds for some k £ No- Fix 5 > 0. We have Vy £ E, 

C+i(y) = (1 - eKP* S N ffi))(y) + e(P' * m£%) . (2.22) 

By Assumption [1] and Lemma [2.1[ we can write, for 5 > 0, 

E(sup \(P'*m%)(y)-(P'*m k )(y)\) = E(sup |(m^ - m k ,P'(.,y))\) 

< E(sup |(mf -m k ,g)\+25) 
geFi 

< ^2E(\{m%-m k ,g)\) + 2S, (2.23) 



where .Fg = t (E). By Lemma [3.7| there exists K(5) compact such that 

Vk(K(8) c ) < 5 , E(^(K(5) C )) <5,\/k,N . 
Using Assumption [3j 21(1) and Lemma |2.U we can write, 

E( sup \(S N (fj£)P(y) -(fj%P)(y)\) 

<E(sup|(S*(^)-^,P(.,y)^ 

< E (\(s N (Vk) - f$,gi m (.))\) + 2^(1 + M P ) 

< 2(^|^ + 6(1 + M P ) ) , (2.24) 



where F% = F^ plp (K(S)), C 4 (S) = \F$\ and the last inequality is a consequence of Lemma 13.5 
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In a similar manner 



E(sup \fj%P(y) - VkP(y)\) < E E (K^ " Vk,9l K (S)(-))\) + 25(1 + M P ) . (2.25) 

geFl 



And so, by (p^2t ([2~23t ([2~24l) . (I2~25l) . 



EdlCi " ^illco) < ( 2CA % MP + 6^(1 + A/p)) + £ E(K 



m fc -m k ,g)\) 

geFl 

+ 2 ^ E(|(r^ -Vk,glK(8))\)- (2-26) 



Recalling that (|1.12p is assumed for k, we have that, as N — > oo, 

E(|lCi-%+illoo) -+0. 
An application of Scheffe's Theorem now shows that, as N — > oo, 

E(||77f +1 - %+1 || T y)^0. 

Next, for any / 6 B\{E), we have 

E(|(mf +1 -m, +1 ,/)|)<E(|mf +1 -mfM^,/)|) 

+ E(|(mfM^fc -mfM^,/)|) +E(|(m^Af* -m fe M^,/)|) 



<^+4AE(||»7f -%||oo)+E(| (mf -m k ,M^f) |), (2.27) 
v W 



where the last inequality uses (|2.1|> and Lemma [3.51 Once again using the recurrence assumption, we now 
have that 

sup E(|(mf +1 -m fc+ i, /)|) —> 0. 
Thus we have proved that (|1.12p holds for fc+ 1. The result follows. □ 
2.5 Proof of Theorem [TTTl 

Proof. For A; € N, we take to be a (random) operator such that ^ +1 {m^f)^) = (ifi^ +1 ,fi^ +1 ). By 

convention we take ^^(mo, r/o) = {^i iVi)- Following the proof of Theorem 11.41 we define 

for 1 < j , $y(m , 770) = $f o o • • • o $ x (m , ?7o) , and for j = , $^(m , r?o) = (m , %)• 
We define VA; G N, 

{ m k,k^k,k) = ®i-.k( m o,Vo), (rn k ,k,Vk,k) = $ $i-.k~i( m o,Vo), 

(™k,k+l>Vk,k+J = H^k,k>Vk,k)A m k,k+i,Vk,k+i) = Hrn ktk ,rik,k)- 

We use the same symbols as in the proof of Theorem 11.41 in order to keep notations simple. We have the 
following telescopic decomposition 

n 

(rh%,r)%) - (m n ,r, n ) = ^[^) ^ i^ l(m0ir|o) _$M) ^ i^ i(mfliI)o) ] . (2 . 28) 

k=i 
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The proof is very similar to the proof of Theorem 11.41 except that now rj kk 7^ T] kk . The strategy remain 

, . — TV — TV 

the same; we want to bound the 'local error term' $^ ' o & l:k _ l — $ o in total variation and then use 
the contraction property of Lemma 12.31 to bound the telescopic sum uniformly in N. Fix (e, A) G (0, 00) 
such that e < eo and A < Ao, where eo, Ao are as in Lemma [2.31 Let 9 = 8(e, A). Also, fix 5 > 0. Consider 
a k G {2, . . . , n - 1}. Note that 

" Vk,k = (1 - e)(S N (vti) ~ ig-i)P. 

Therefore, from fl2.24j) 



H\Kk ~ Vk,k\\oo) < 2 ( * K ' r - + 5(1 + M P )) . (2.29) 



In the same way as (|2.2ip in the proof of Theorem 11.41 (see also (|2.20p ). we get 



E(\\m£ k M^ - m Kk M^ \\ TV ) < 2 ( 8 + ) . (2.30) 

V A 



Also from (|2.ip and Lemma 12.21 we have V/ G Si (P) 

|<m£ fc M<* -m&M**.*,/ > | < 4||^ - r^U . (2.31) 
Equations fl2~29j) . (l2~30l . (|2~3Tj) yield 



E(||< fe+1 " m M+1 || T v) < 8g 4fflMp + 2C 2W + d{w + ^ (2 32) 

v A 



Next, with X(<5) as in the proof of Theorem [L6l V/ G £i(K d ) 

< |(^(f7f_ 1 )-Cl^/(0lK(5)(0)| + l^(Cl)(^) C )l (2-33) 

+id(^w c )i- 

Also, using the Gaussian property of the kernel P, it follows that Pf is Lipschitz on K(5), uniformly in 
/ G ,61 (M d ). Denote the uniform bound on the Lipschitz norm by Ik(S)- Then 

\(S N (fj»)-fj»,Pfl K{s) )\ < max\(S N (fj£)-rj»,gl K{5) )\ + 26, (2.34) 

geF§ 

where F* = Pf^ g (K(S)). Thus 

E|< fc (/) - %,*(/)! < EmaxK^CCi) " Ci,<?l*(5))l 



Next, for all / G £j 

< E| « fe - %, fc , P/) I + E| <m£ fe P' - m kik P>, f) \ 



< E(max K^^l) " + 45 + E||m^P' - m fc , fc P'|| T v • 

Using C\{5) introduced in the proof of Theorem 11.41 we have 

E(||m^P - m k , k P'\\ TV ) < 2 (5 + . 
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We then get 



H\A +1 -v k , k+ i\\ T v) < 2 ^+ Cl( ^ 6W ) 

where C 6 (6) = \F$\. Also, by (l2~33l) 



H\\rit-Vn,n\\Tv)<2(26+^j 



and using the definitions of m„ n and m n;n , we get from Lemma 1331 

2 

sup E(\(m nn - m„, n ,/)|) < 



Thus as in the proof of Theorem 11,41 we get, for n > 1, 
sup E(|(m^ - + ||?7^ - 77 n || Ty ) 

< » + 2 f M + 1 + o fa + M f) + 2C ' w + 2C * w + 2C ° <*> + 8C * w ) + ^. 

"VI \ \ y/N J i-e 



□ 



3 Appendix: auxiliary results 

Lemma 3.1. Let M be a transition probability kernel on E such that for some e G (0, 1) and l\ G V{E), 

M(x,A) > e£i(A),V A G B(E),x G £7. 

Then, for all /ii,/i2 G V{E), 

WmM - ll 2 M\\tv < (1 - e)||w - M2II2V. 

Proof. This result comes from Dobrushin's Theorem and the proof of this theorem can be found in [T], p. 
183 and p. 192 (see useful definitions on p. 181). Dobrushin's Theorem can also be found in [4], p. 70, 
with the corresponding proof in [5j, p. 332. □ 

Lemma 3.2. Suppose that Assumption^ holds. Then, for any m,m' G V(M. d ) and 77 G V*(M. d ), 



ImM* 1 - rn'M^Wrv < (1 - e Q e 



-xosc( v ) Mlm _ m > 



Proof. The proof can be found in [T], p. 195. Since this book is written in French, we give a quick proof. 
Note that Vx, y G E, 

M^x, dy) = Q(x, dy)e- XMx) -^ v)) + + Q (x, dy) (l - Q( x , dz)e~ x ^~^+ ^ . 
Then by Assumption [2] 

>e Q e- X0SC ^h(dy). 

The result now follows from Lemma 13.11 □ 



Lemma 3.3. Let rj G V(M. d ), m G V(E), and let rj = rj'R m . Suppose that Assumption^ (3) holds. Then, 
r\ has a lp t pi- Lipschitz density. 
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Proof. Under Assumption U] (3) , the density of rj is 

x\->(l-e) T)'(dy)P(y,x) + e m{dy)P(y,x) 

Ji/GM d Jv€R d 



And for x, x' £ R d , 

\rj(x) — r](x')\ < (1 — e) / v'(dy)lp t pi\x — x'\ + e / m(dy)lp y pi\x — x'\ 

= Ip pi\x — x \ . 



□ 



Lemma 3.4. Take X] € V(M. d ) having a l„-Lipschitz density and let f E Bi(K d ). Under Assumption^ the 
function x E E h- > M 1] f(x) is lQ t Q -Lipschitz with 

1q,Qo = h,Qo ( 3 + 2 ^ l v) ■ 

Proof. Note that for x G E 



M r, f{x)= / Q(x,dy)e- XMx) -^ y)) +f(y) 



y&E 



+ / Q (x,dy)f(y) x ( 1 

ly&E 



Q{x,dz){l - e~ x ^ x) -^ z)) + ] . 



2e£ 



So for G E, using Assumption [5] 
\My{x)-M^{x')\<l Q , Qo \x-x'\ 



+ / Q(x',dy)\e- XMx) -^ y)) + - e- XMx,) - viy)) +\ 



+ 



(Qo(x,dy) - Q {x' ,dy))f{y) 



y€E 



X 1 



eE 



Q(x,dz)(l-e- x ^-^+ 



+ / Q (x',dy) x 

Jy&E 



eE 



Q(x,dz)e- x ^-^+ - Q(x\dz)e- X( ' n(x '^ z ^ 



+ l Q,Qo\ x ~ x '\+ / Q{x',dy)X\rj{x) -r](x')\ 

JydE 



</Q, Qo |x-x , |(3 + 2Ag 



Lemma 3.5. (1) Let Eq be a closed subset ofM. d and let p, G V{Eq). Then for all f E Si(£'o) 



□ 



e(|(^(m)-m,/>I)< 



A 



^ Lei G be a transition probability kernel on a closed subset Eq ofM. d . Let x\,---xn G -E"o / or so^ie 
A G N and let £i, • • • £/v 6e mutually independent random variables distributed as 5 X1 G, ■ ■ ■ 5 XN G. Define 
m o = w Ei=i ^* anrf m i = F £i=i % ■ T/ien / or °^ / 6 ^i(^o) 



E(|(mf-m^G,/}|)< 



A 
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Proof. We will only show (2). Proof of (1) is similar. Fix / € Bi(Eq). Then 

N 

E(|(mf -m$GJ)\) = E 



^ (e(^E/«*)-m?(/)) 
= ( E ^E(/&)-^ G (/)) 2 



2\ 1/2 



1/2 



< 



N 



□ 



Lemma 3.6. Suppose P,P' satisfy Assumption® then they satisfy Assumption\3{l) . 
Proof. Let mQ,m' £ V{E). We can write 

{mo = m + rh 
m = m + rh' 

where m,m,m' are finite measures on E and ||mo — m ollrv = ll m ~~ ^'[Itv = ll m ll + II^'IItv- We set 

(p' E (x,dx>) = P'(x,dx')l x , eE 
{P' EC (x,dx')= P'{x,dx')l x , eE c 

Since E is compact, we can find p 6 V(E) and J > such that 

P' E (x, dx') = Sp(dx') + M(x, dx') , Vx G E. 

where M is some nonnegative kernel. Then 

\\moP' — m' P'\\Tv = \\mP E ~ ™' Pe\\tv + \\™Pe c ~ ™' Pe c \\tv 
< \\rhM - rh'M\\ TV + rhP'(E c ) +m'P'(E c ) 



(3.1) 



From (|3.ip we have 



< rhM{E) + fh'M(E) + mP'(E c ) + rh'P'(E c ) . 

rhM(E) + mP'(E c ) = rhP'(E) - Srh(E) + mP'(E c ) 
< (1 - 5)mP'(E) + mP'(E c ) 
= (l-5)rhP'(E) + (rh(E)-rhP'(E)). 

Since E has a non-empty interior, we can find a > such that Vm G V(E), TnP'(E) > a. In particular 
we have rhP'(E) > arh(E). Using this property in the above display 

rhM(E) + mP'(E c ) < (1 - 5)am(E) + (1 - a)rh(E) 
= (1 - a5)m(E) , 

Also, the same formula holds with rh replaced by rh'. So 

\\m P' -m' P'\\ T v < (1 - a5)(m(E) + rh'{E)) 
= (1 - a6)\\m - m' \\ T v ■ 

□ 



19 



Lemma 3.7. Suppose that Assumption |^] (0|) holds. Fix (mo,<7o) £ 7*(E) X V*(M. d ). Let rjk be as defined 
in (jl,9p and 
i/iai Vfc, ViV, 



^n (|l,9p anc? Zet 77^ 6e as defined in f| 1 . 1 1 [) . ITien, /or any o" > 0, 3 a compact subset of~U. d , K{5), such 



m (K(6) c )<6,E(fj%(K(6T)<6. 
Proof. We can prove by recurrence that Vn, 



n 

^ [e(l - e) k m n „ k P'P k ] + (1 - ef+^oP' 



yn+l 



Vn+1 

k=0 

Fix 5 > 0. Let ko such that Sfc>fc +i(l — e ) k < ^- Since E 1 is compact, we can find a finite family (xj)j^j 
of elements of E 1 such that \/y G 3j G J such that |y — Xj\ < 5. We can partition: E = Uj^jEj such 
that Vj and Vy G |y — ajj| < 5. For each j G J, we can find if,-, a compact subset of M. d , such that 
(i, ,/"(A'J) < J, 8 Xj P'P{K c j ) <8,.. . ,5 X] P'P ko {K c j ) < 8. Clearly, the set if = U ie j^ is compact. Also, 
yk < ko, Vr 



m r P'P k (K c ) = [ m r (dy)(P'P k )(y,K c ) 

Jy&E 

= E/ m r (dy)(P'P fc )(y J Jn 



= E/ m r (d V )(P'P k )(.x j ,K< : ) 
jeJ J y eE i 

+ E / m r (^)((P'P fe )(y,K c ) - (P'P fe )(x i; K c )) 
< Y,[^(E 3 )(P'P k )(x 3 ,K^ 



< 5 + Slp^p/ , 

where the next to last inequality follows from Assumption [4] (2). Finally, Vn > ko + 1 

k=0 

ko 

< Y,[ e (l-e) k (6 + 6ip,p,)] + 6. 

k=0 

The first statement follows. To prove the second statement, we begin by defining the (random )operator 
S N oP acting on probability measures on fi(S N o P) = (S N (fi))P. We have 

n 

f}% +1 = E - ^rr^P'^ o P) fc ] + (1 - 6)"+ 1 r ?0 (^ o Pf+* . 

k=0 

We take the same 5, Xj,Kj, ... as above and we notice that \/k, j, 

E(8 Xj P'(S N o P) k (Kf)) = 8 X] P'P k {K c J ) . 
From this point, the proof is the same as for the first statement. □ 

Acknowledgement s 

AB has been supported in part by the National Science Foundation (DMS-1004418), the Army 
Research Office (W911NF-0-1-0080, W911NF-10-1-0158) and the US-Israel Binational Science Foun- 
dation (2008466). Part of this work was done when AB was visiting Universite de Nice-Sophia Antipolis 
and Universite de Bordeaux I and when SR was visiting University of North Carolina at Chapel Hill. 
Hospitality of these universities is gratefully acknowledged. 



20 



References 

[1] Nathalie Bartoli and Pierre Del Moral. Simulation & Algorithmes Stochastiques. Cepadues editions, 
2001. 

[2] Frangois Caron, Pierre Del Moral, Arnaud Doucet, and Michele Pace. Particle approximations of 
a class of branching distribution flows arising in multi-target tracking. Research Report RR-7233, 
INRIA, 03 2010. 

[3] Francois Caron, Pierre Del Moral, Michele Pace, and Ba-Ngu Vo. On the Stability and the Approx- 
imation of Branching Distribution Flows, with Applications to Nonlinear Multiple Target Filtering. 
Stochastic Analysis and Applications, To appear, 2010. 

[4] R. L. Dobrushin. Central limit theorem for nonstationary Markov chains I. Theory Probab. Appl., 
l(l):65-80, 1956. 

[5] R. L. Dobrushin. Central limit theorem for nonstationary Markov chains II. Theory Probab. Appl., 
l(4):329-383, 1956. 

[6] Avner Friedman and J. Ignacio Tello. Stability of solutions of chemotaxis equations in reinforced 
random walks. J. Math. Anal. Appl., 272(1):138-163, 2002. 

[7] Carl Graham and Philippe Robert. Interacting multi-class transmissions in large stochastic networks. 
Ann. Appl. Probab., 19(6):2334-2361, 2009. 

[8] Carl Graham Javier Gomez-Serrano and Jean yves Le Boudec. The bounded confidence model of 
opinion dynamics. 2010. 

[9] B. Latane and A. Nowak. Self-organizing social systems: Necessary and sufficient conditions for the 
emergence of clustering, consolidation, and continuing diversity. Progress in communication sciences, 
pages 43-74, 1997. 

[10] Frank Schweitzer. Brownian agents and active particles. Springer Series in Synergetics. Springer- 
Verlag, Berlin, 2003. Collective dynamics in the natural and social sciences, With a foreword by J. 
Doyne Farmer. 

[11] A. Stevens. Trail following and aggregation of myxobacteria. J. Biol. Syst., (3):1059-1068, 1995. 

[12] Alain-Sol Sznitman. Topics in propagation of chaos. In Ecole d'Ete de Probabilites de Saint-Flour 
XIX— 1989, volume 1464 of Lecture Notes in Math., pages 165-251. Springer, Berlin, 1991. 



21 



